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Abstract 


This  research  proposes  a  method  for  automatically  designing  sensors  from  the  specification  of 
a  robot's  task,  its  actions,  and  its  uncertainty  in  control.  The  sensors  provide  precisely  the 
information  required  by  the  robot  to  perform  its  task,  despite  uncertainty  in  sensing  and 
control.  The  key  idea  is  to  generate  a  strategy  for  a  robot  task  by  using  a  backchaining 
planner  that  assumes  perfect  sensing  while  taking  careful  account  of  control  uncertainty. 
The  resulting  plan  indirectly  specifies  a  sensor  that  tells  the  robot  when  to  execute  which 
action.  Although  the  planner  assumes  perfect  sensing  information,  the  sensor  need  not 
actually  provide  perfect  information.  Instead,  the  sensor  provides  only  the  information 
required  for  the  plan  to  function  correctly. 


This  report  is  a  revised  version  of  a  proposal  currently  submitted  to  NSF. 
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1  Project  Summary 

This  research  proposes  a  method  for  automatically  designing  sensors  from  the  specification 
of  a  robot’s  task,  its  actions,  and  its  uncertainty  in  control.  The  sensors  provide  precisely 
the  information  required  by  the  robot  to  perform  its  task,  despite  uncertainty  in  sensing  and 
control.  The  key  idea  is  to  generate  a  strategy  for  a  robot  task  by  using  a  backchaining  planner 
that  assumes  perfect  sensing  while  taking  careful  account  of  control  uncertainty.  The  resulting 
plan  indirectly  specifies  a  sensor  that  tells  the  robot  when  to  execute  which  action.  Although 
the  planner  assumes  perfect  sensing  information,  the  sensor  need  not  actually  provide  perfect 
information.  Instead,  the  sensor  provides  only  the  information  required  for  the  plan  to  function 
correctly. 

In  the  past,  models  of  sensing  have  been  driven  from  the  bottom,  by  questions  of  the  form 
“What  are  the  characteristics  of  sensor  X?”,  “What  is  the  error?”,  “How  can  we  use  the 
sensor?”.  We  are  proposing  models  of  sensing  that  are  driven  from  the  top,  by  questions  of 
the  form  “Given  a  task,  what  sensor  characteristics  are  required?”,  “How  much  uncertainty 
can  be  tolerated?” ,  “How  does  sensing  depend  on  action?” . 

This  research  will  investigate  special  purpose  sensors  for  simple  assembly  tasks.  These 
tasks  include  multiple-peg-in-hole  assembly,  parts  identification,  parts  orienting,  and  parts 
sorting.  The  long-range  goal  of  this  research  is  to  develop  task-level  planners  capable  of 
translating  high-level  specifications  into  low-level  robot  commands.  Sensor  design  is  a  means 
of  understanding  the  information  requirements  of  manipulation  tasks  and  thus  a  means  of 
improving  task  specifications.  In  addition,  sensor  design  is  one  approach  for  breaking  the 
complexity  barrier  that  impedes  practical  planning  in  the  presence  of  uncertainty.  This 
research  could  lead  to  improved  parts  design,  improved  sensor  design,  and  more  efficient 
automatic  programming  tools. 
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2  Introduction 


The  goal  of  robotics  is  to  develop  autonomous  systems.  In  order  to  achieve  this  goal,  we  must 
understand  the  relationship  between  action  and  sensing,  and  we  must  be  able  to  convert  that 
understanding  into  robots  capable  of  operating  productively  and  successfully  in  an  uncertain 
world. 

The  nature  of  the  relationship  between  action  and  sensing  raises  some  fundamental 
questions: 

•  What  is  the  information  needed  to  solve  a  given  manipulation  task? 

•  What  tasks  can  be  solved  by  a  given  repertoire  of  operations? 

•  How  sensitive  are  solutions  of  tasks  to  particular  assumptions  about  the  world? 

Obtaining  answers  to  these  questions  forms  the  central  research  agenda  in  robotics.  In 
this  proposal  we  focus  on  the  first  question. 

Our  goal  in  pursuing  this  line  of  research  is  to  improve  and  automate  robot  programming 
tools.  Our  long-range  research  agenda  is  to  develop  a  high-level  representation  of  tasks  that 
facilitates  the  decomposition  of  robot  tasks  into  modular  subtasks  and  the  translation  of 
high-level  specifications  into  low-level  robot  commands.  This  translation  and  decomposition 
process  is  severely  complicated  by  uncertainty.  We  review  some  of  the  relevant  work  in 
Section  7.  Given  uncertainty,  our  current  best  approach  for  improving  robot  programming 
and  developing  task- level  planners  is  to  understand  the  information  requirements  of  robot 
tasks. 

We  propose  a  method  for  connecting  action  to  sensing,  by  designing  sensors  particularly 
suited  to  the  actions  available  to  the  robot.  The  sensors  provide  precisely  the  information 
required  by  the  robot  to  function,  despite  uncertainty.  This  approach  stands  in  contrast  to 
the  idea  of  using  available  sensors  and  manipulators  to  accomplish  a  given  task.  Indeed,  the 
approach  in  this  proposal  is  to  control  sensing  as  one  controls  action. 

Although  the  problem  of  controlling  sensing  and  action  may  seem  to  be  more  complicated 
than  the  problem  of  controlling  only  action,  by  using  some  simple  guiding  principles,  this 
extra  degree  of  controllability  can  actually  simplify  the  problem  of  planning  and  executing 
manipulation  strategies  in  the  presence  of  uncertainty.  The  primary  principle  is  to  generate  a 
strategy  that  tightly  integrates  action  and  sensing  by  first  temporarily  decoupling  action  from 
sensing.  The  key  idea  is  to  generate  a  strategy  for  a  given  task  by  using  a  backchaining  planner 
that  assumes  perfect  sensing  but  that  accounts  for  control  uncertainty.  Given  such  a  strategy, 
one  then  designs  a  sensor.  Although  the  planner  assumes  perfect  sensing  information,  the 
sensor  provides  only  the  information  required  for  the  strategy  to  function  correctly,  despite 
noise  in  the  sensor  and  uncertainty  in  control. 

3  What  is  the  information  needed  to  solve  a  task? 

This  question  is  a  most  important  question  in  robotics.  Within  it  are  contained  the  basic 
issues  of  task  definition,  action  specification,  environment  design,  and  sensing  capabilities. 
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3.1  Repositories  of  Information 

When  we  ask  “what  information  is  required?”  some  answers  seem  to  appear  immediately.  Yet, 
when  we  look  more  closely  these  answers  become  fuzzy.  The  answers  that  appear  immediately 
axe  of  the  form:  some  information  arises  from  sensors,  other  information  is  encoded  in 
the  mechanism  performing  the  task,  some  information  is  encoded  in  the  mechanism’s 
interpretation  of  the  world,  some  in  its  predictive  ability,  and  still  other  information  acts 
subtly  through  the  environment.  It  is  obvious  why  these  answers  are  fuzzy.  They  are  fuzzy 
because  there  is  no  clear  notion  of  how  the  information  is  distributed  between  these  different 
repositories  of  information. 


3.2  A  Bewildering  Array  of  Strategies 

As  an  example,  imagine  the  task  of  placing  a  pencil  into  an  electric  pencil  sharpener  in  order  to 
sharpen  the  pencil.  There  are  numerous  methods  for  accomplishing  this  task.  One  is  to  grasp 
the  pencil  then  move  one’s  hand  slowly  while  carefully  looking  at  the  pencil  and  the  sharpener, 
continuously  readjusting  one’s  aim  as  the  pencil  comes  close  to  the  sharpener’s  hole.  Another 
is  to  close  one’s  eyes,  place  one  hand  on  the  pencil  sharpener,  then  with  the  other  hand  bring 
the  pencil  to  the  sharpener,  and  guide  it  manually  into  the  hole  using  tactile  feedback.  A 
third  strategy  is  to  visually  memorize  the  state  of  the  world  then  move  the  pencil  into  the 
sharpener  with  no  feedback  whatsoever.  A  fourth  strategy  consists  of  randomly  stabbing  at 
the  sharpener  with  the  pencil  until  the  pencil  enters  the  sharpener.  A  fifth  strategy  consists  of 
building  a  linkage  of  some  sort  that  can  insert  the  pencil  into  the  sharpener  from  a  large  set  of 
initial  states.  The  linkage  is  specially  designed  as  a  function  of  the  sharpener’s  geometry  and 
the  possible  starting  locations  of  the  pencil.  A  sixth  strategy  is  to  dispense  with  the  notions 
of  pencil  and  sharpener,  and  instead  build  a  series  of  tightly  coupled  sensory  feedback  loops, 
implemented  on  top  of  hardware  that  can  grasp  and  move  long  thin  objects.  The  feedback 
loops  might  implement  such  operations  as  pencil-seeking,  pencil-grasping,  and  hole-seeking. 
Numerous  other  strategies  may  be  constructed  as  hybrids  of  these  six  strategies. 

This  enormous  array  of  strategies  leaves  one  with  a  distinct  feeling  of  unease.  In  particular, 
the  strategies  seem  to  gain  their  information  very  differently.  Some  rely  heavily  on  sensors, 
others  on  mechanical  devices,  and  yet  others  on  prediction.  This  makes  it  very  difficult  to 
answer  the  question  “what  information  is  required  to  solve  the  task  of  putting  a  pencil  into 
a  sharpener?”  Indeed,  we  could  complicate  the  question  even  more  by  varying  the  sizes  and 
shapes  of  the  pencil  and  the  sharpener.  One  is  tempted  to  dismiss  the  question  as  non-sensical, 
based  on  the  apparent  realization  that  there  is  no  intrinsic  measure  of  required  information 
and  even  no  intrinsic  task  of  the  form  Put  THE  PENCIL  IN  THE'  SHARPENER. 

3.3  Designing  Sensors  for  Tasks 

We  will  resist  this  temptation,  and  instead  introduce  time  as  a  measure  relative  to  which  one 
can  study  information  requirements.  Specifically,  one  can  first  seek  the  information  required 
to  accomplish  a  task  as  quickly  as  possible.  Relative  to  this  base  standard  one  can  then 
compare  other  strategies  and  their  information  requirements. 

A  related  approach  is  to  realize  that  while  all  the  strategies  mentioned  above  are  defined 


relative  to  a  given  world  and  a  given  repertoire  of  actions,  the  strategies  have  different 
applicability  as  one  varies  these  worlds.  In  particular,  if  one  increases  uncertainty,  or  if 
one  changes  the  tolerances  of  the  world,  or  if  one  adds  new  features  to  the  world,  then  some 
strategies  will  remain  in  effect,  while  others  will  falter.  The  degree  to  which  the  original  world 
may  be  be  modified  gives  a  measure  of  the  true  information  requirements  of  the  task  and  thus 
lends  substance  to  the  task  abstraction.  Equivalently,  the  extent  to  which  information  must 
be  added,  either  by  adding  new  sensors  or  mechanically  constraining  the  motions  of  the  robot 
in  the  world,  gives  a  measure  of  the  intrinsic  nature  of  the  task.  This  reasoning  thus  also 
leads  us  to  the  idea  that  one  should  study  the  problems  of  designing  sensors  and  of  designing 
mechanical  devices  that  provide  precisely  the  information  required  to  accomplish  a  task  with 
a  given  repertoire  of  actions  in  a  given  amount  of  time.  In  this  proposal  we  focus  on  sensor 
design. 

4  Complexity 

This  section  argues  for  sensor  design  based  on  an  examination  of  the  complexity  of  several 
different  approaches  to  motion  planning.  We  draw  our  complexity  estimates  from  two  domains. 
One  domain  consists  of  discrete  state  spaces  with  discretely  specified  actions.  The  actions 
themselves  have  non-deterministic  or  probabilistic  transitions.  See  [Papadimitriou  1985], 
[Papadimitriou  and  Tsitsiklis  1987],  [Bertsekas  1987],  and  [Erdmann  1990]  for  further  details. 
The  other  domain  consists  of  continuous  spaces  with  continuous  actions,  analogous  to  those  in 
the  LMT  [Lozano-Perez,  Mason,  and  Taylor  1984]  preimage  methodology.  Again,  the  actions 
are  either  non-deterministic  or  probabilistic.  See  [Natarajan  1986,  1988],  [Canny  and  Reif 
1987],  [Canny  1988a,  1989],  and  [Donald  1988b]  for  further  details. 

4.1  Planning  with  Perfect  Sensing  and  Uncertain  Control  is  Easy 

Table  1  summarizes  the  basic  results.  We  omit  many  details.  One  notices  the  following 
pattern:  for  perfect  sensing,  planning  seems  to  be  easy;  for  general  sensing,  planning  seems 
to  be  intractable;  and  for  sensorless  strategies,  planning  seems  to  be  hard,  but  not  as  hard  as 
in  the  general  case. 

The  interesting  observation  is  that  planning  with  perfect  sensing  is  usually  easy,  even  in 
the  presence  of  control  uncertainty.  Indeed  the  complexity  of  deciding  the  existence  of  a 
strategy  for  attaining  the  goal  with  uncertain  control  seems  to  be  nearly  that  of  determining 
the  connectivity  of  the  state  space. 


4.2  Design  Philosophy 

Previous  work  on  planning  in  the  presence  of  uncertainty  has  considered  the  general  problem, 
in  which  both  sensing  and  control  are  subject  to  uncertainty,  and  the  special  problem,  in  which 
sensing  is  completely  absent  and  control  is  uncertain.  These  problems  correspond  to  the  top 
two  rows  of  Table  1.  These  problems  naturally  lie  in  an  intractable  complexity  class.  The 
results  of  Table  1  suggest  that  it  makes  sense  to  look  at  problems  in  which  sensing  is  perfect. 
One  is  reluctant  to  do  so,  since  perfect  sensing  is  impossible.  There  is  a  much  more  attractive 


6 


Sensorless 
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NP 

2(nJfc)°<1> 

Sensing 

Uncertainty 

Imperfect 

PSPACE 

PSPACE 

22°<n*> 

Perfect 

P 

P 

P 

Perfect 

Probabilistic 

Adversarial 

Discrete; 

Optimal 

Discrete; 

Optimal 

Continuous; 

Guaranteed 

Control  Uncertainty 

Task  Space; 
Strategy  Type 

Table  1:  A  rough  complexity  picture.  In  each  case  the  problem  is  to  decide  the  existence  of  a 
Ar-step  strategy  for  attaining  the  goal  from  some  initial  region.  The  right-most  column  concerns 
tasks  on  continuous  spaces,  with  the  objective  of  finding  a  guaranteed  strategy  that  attains  the 
goal.  The  other  two  columns  concern  tasks  on  discrete  spaces,  with  the  objective  of  obtaining  an 
optimal  strategy.  The  complexity  classes  are  classifications,  except  for  the  two  exponentials,  which 
are  known  upper  bounds.  The  number  of  obstacle  constraints  is  n.  (“Perfect”  means  that  there  is 
no  uncertainty  of  that  type;  “Adversarial”  refers  to  unknown  but  bounded  errors.) 

view,  however.  This  view  envisions  sensing  as  a  design  problem.  For  any  task,  one  can  assume 
perfect  sensing  in  determining  the  control  strategy  for  accomplishing  that  task.  Having  done 
so,  one  then  builds  an  imperfect  sensor  that  nonetheless  provides  the  information  required  for 
this  control  strategy  to  operate.  One  big  research  issue  is  understanding  the  degree  to  which 
degradation  of  information  still  permits  the  control  strategy  to  operate.  Another  research 
issue  to  decide  whether  an  abstract  sensor  generated  by  this  design  process  may  be  built, 
both  geometrically  and  physically. 

5  A  Family  of  Sensors  for  the  Point-Into-Disk  Problem 

The  basic  thesis  of  this  proposal  is  that  one  should  explore  the  special  purpose  design  of 
sensors  based  on  a  system’s  available  actions,  in  order  to  more  easily  accomplish  a  given 
manipulation  task  in  the  presence  of  uncertainty.  The  encompassing  research  objective  is  to 
obtain  an  understanding  of  the  knowledge  requirements  of  manipulation  tasks. 

In  this  section  we  illustrate  these  goals  with  a  simple  task.  The  task  is  to  move  a  point 
in  the  plane  into  a  circular  disk  centered  at  the  origin  of  the  plane.  We  will  look  at  the 
design  of  a  sensor  for  this  task,  that  is  specifically  structured  to  guide  the  point  into  the  disk. 
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In  the  process,  we  will  see  that  it  is  possible  to  design  a  family  of  sensors,  indexed  by  the 
speed  with  which  the  point  approaches  the  goal.  The  family  of  sensors  defines  a  tradeoff 
between  the  optimal  approach  strategy  that  is  possible  with  perfect  sensing  and 
the  motions  possible  given  the  actual  information  available  from  an  imperfect 
sensor. 

Let  us  note  in  passing  that  the  point-into-disk  problem  is  the  archetypical  abstraction  of 
the  assembly  problem.  Whenever  two  parts  must  be  mated,  the  problem  can  be  viewed  as  a 
point-into- volume  problem  in  some  appropriate  parameter  space  [Lozano-Perez  1981,  1983]. 
With  this  configuration  space  transformation  in  mind,  we  sometimes  refer  to  the  moving  point 
as  a  “robot”. 

5.1  Actions  and  Uncertainty 

Let  us  define  the  actions  available  for  the  point-into-disk  problem.  The  system  is  a  simple 
first-order  system,  in  which  the  point  is  moved  by  commanding  velocities  that  are  subject 
to  unknown  but  bounded  errors.  Specifically,  if  the  execution  system  commands  velocity  v0, 
then  the  range  of  possible  velocities  is  given  by  the  ball  Btv  |v0|(vo)>  where 

Btv  |Vo|(v0)  =  {v*  |  |v0  -  v*|  <  e„  |v0|}. 

Another  often  useful  way  to  think  of  velocity  uncertainty  is  as  a  cone  of  possible  velocity 
directions.  For  a  nominal  commanded  velocity  v0  and  error  radius  tv  |v0|,  the  cone  has  axis 
given  by  v0,  and  error  angle  sin-1(e„). 

A  basic  action  consists  of  commanding  a  velocity  v0  for  some  duration  of  time  At,  where 
At  can  be  arbitrarily  small.  This  model  of  action  resembles  closely  the  idea  that  the  executive 
system  will  in  a  nearly  continuous  fashion  execute  a  velocity  for  a  small  amount  of  time, 
determine  the  robot’s  new  location,  then  execute  a  new  velocity. 

Now,  suppose  that  the  initial  position  of  the  point  is  known  to  lie  in  some  set  R ,  and 
suppose  that  one  commands  velocity  v0  for  time  At.  Assuming  no  obstacles,  the  resulting 
locations  of  the  point  are  described  by  the  set  sum  R  ®  Btv  |Vo|  At(vo  At).  In  other  words,  each 
possible  initial  location  is  offset  by  a  possible  velocity  scaled  by  the  elapsed  time. 

5.2  An  Ideal  Sensor 

Let  us  ask  ourselves  what  the  ideal  sensor  for  the  point-into-disk  task  might  look  like.  A  sensor 
that  reports  precisely  the  current  position  of  the  point  would  certainly  be  nice.  In  particular, 
given  a  position  reported  by  this  sensor,  the  execution  system  can  calculate  a  velocity  vector 
that  aims  directly  towards  the  goal.  We  will  see  presently  that  this  perfect  position  sensor  is 
actually  superideal,  in  that  it  provides  more  information  than  is  needed. 

5.2.1  A  World-Centered  Sensor:  Poor  Degradation  with  Noise 

One  issue  to  keep  in  mind  is  how  one  might  implement  a  perfect  position  sensor.  One 
possibility  is  to  have  a  physical  sensor  that  estimates  the  robot’s  position  by  observing  it. 
A  camera  is  a  good  example.  Another  implementation  might  consist  of  a  series  of  encoders 
attached  to  the  control  system  for  the  robot.  This  is  a  form  of  dead-reckoning.  A  third 
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implementation  might  consist  of  a  grid  of  rectangular  wires  implanted  in  the  plane,  that 
report  back  the  ( x ,  y)  position  of  the  robot. 

Observe  that  all  of  these  sensors  are  world- centered.  Specifically,  they  report  back  the 
configuration  of  the  robot  in  some  world  coordinate  system.  Given  this  configuration,  the 
run-time  executive  then  calculates  the  difference  in  position  between  the  robot  and  the  goal  in 
order  to  suggest  a  commanded  velocity.  This  procedure  is  fine  for  a  truly  perfect  sensor,  but 
degrades  poorly  as  the  sensor  becomes  noisy  or  as  the  position  of  the  goal  becomes  uncertain. 
In  particular,  suppose  we  model  the  uncertainty  of  such  a  world-centered  sensor  as  an  error 
ball  with  a  fixed  error  radius.  If  the  error  radius  is  larger  than  the  goal,  then  as  the  robot 
approaches  close  to  its  goal,  the  system  will  be  unable  to  decide  on  which  side  of  the  goal  the 
robot  is  located.  This  makes  it  impossible  to  execute  a  velocity  guaranteed  to  at  ain  the  goal. 

While  this  example  is  simple,  the  problem  is  fundamental.  It  exists  in  even  greater 
magnitude  for  general  manipulation  tasks.  In  these  tasks  the  information  available  to  the 
robot  may  consist  not  just  of  positional  information,  but  also  of  information  coming  from 
such  sources  as  force  sensors,  history,  and  model-based  prediction.  It  is  the  interaction  of  all 
these  sources  of  world-centered  information  that  makes  the  planning  problem  so  difficult. 

5.2.2  A  Task-Centered  Sensor:  Respects  Uncertainty 

A  second  approach  is  to  build  a  task-centered  sensor.  What  might  such  a  sensor  for  the  point- 
into-disk  task  look  like?  A  good  representation  for  a  perfect  sensor  is  not  a  cartesian  grid 
but  a  polar  coordinate  grid  that  is  centered  at  the  goal.  As  with  the  perfect  cartesian  sensor, 
the  perfect  polar  coordinate  sensor  reports  back  the  configuration  of  the  robot  relative  to  the 
hole,  but  in  polar  coordinates.  The  coordinates  are  then  again  used  to  compute  a  velocity 
vector  along  which  the  robot  should  move. 

Fortunately,  we  can  go  yet  a  step  further.  The  polar  coordinate  sensor  reports  back  both 
the  distance  of  the  robot  from  the  goal  and  the  angle  that  measures  the  relative  direction 
between  the  robot  and  the  goal.  Only  the  angle  is  used  to  calculate  a  motion  command; 
the  distance  is  ignored.  This  observation  immediately  suggests  that  the  ideal  sensor  need 
only  report  the  angular  coordinate  of  the  polar  coordinate  representation.  In  short,  the  ideal 
perfect  sensor  is  a  directional  beacon  situated  at  the  goal,  that  reports  the  direction  from  the 
goal  to  the  robot.  This  sensor  senses  one  degree  of  informational  freedom. 

Let  us  observe  that  this  task-centered  sensor  degrades  nicely  with  noise.  In  particular, 
an  error  of  a  few  degrees  in  the  directional  beacon  simply  means  that  the  robot  will  move 
slightly  in  the  wrong  direction.  Near  the  goal,  the  sensor  can  quickly  detect  and  correct  such 
inaccurate  motions.  In  contrast,  for  the  cartesian  sensor,  slight  positional  errors  near  the  goal 
can  lead  to  large  directional  uncertainties  that  remain  undetected  for  long  times. 

There  are  two  principles  at  play  here.  First,  the  sensor  is  task-centered.  Second,  the 
sensor  error  mirrors  the  control  error.  We  will  explain  this  second  principle  in  the  next  few 
paragraphs. 

5.3  Designing  a  Sensor  with  Time-Indexed  Backprojections 

The  basic  procedure  for  designing  both  the  sensor  and  the  strategy  that  together  accomplish 
the  task  is  to  backchain  from  the  goal  under  the  assumption  of  a  perfect  sensor.  The  reason  for 
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Figure  1:  This  figure  shows  a  typical  backprojection  of  a  two-dimensional  disk. 

using  a  perfect  sensor  is  that  it  considerably  simplifies  the  planning  problem,  without  forcing 
us  into  a  particular  implementation  of  the  sensor.  The  backchaining  process  tells  us  what 
information  is  required,  how  it  is  used,  and  how  to  design  a  sensor  so  that  the  information 
degrades  nicely  in  the  presence  of  uncertainty. 

We  can  describe  the  backchaining  process  inductively.  Initially,  the  planner  constructs  a 
collection  of  backchaining  regions  from  the  goal  disk.  At  each  subsequent  stage,  the  planner 
constructs  a  collection  of  backchaining  regions  from  some  other  circular  region.  The  semantics 
are  that  at  each  stage  there  is  some  disk  within  which  a  strategy  for  accomplishing  the  goal 
in  the  presence  of  perfect  sensing  and  imperfect  control  has  been  constructed.  The  job  of  the 
planner  is  to  enlarge  this  strategy  to  a  slightly  larger  disk. 

In  order  to  understand  the  operation  of  the  planner,  consider  first  the  problem  of  forming 
a  backprojection  of  a  circular  disk  relative  to  some  commanded  velocity.  See  [Erdmann  1984, 
1986].  In  brief,  the  backprojection  of  a  goal  relative  to  some  commanded  velocity  consists  of 
those  configurations  from  which  the  commanded  velocity  is  guaranteed  to  cause  the  robot  to 
pass  through  the  goal,  despite  uncertainty  in  the  actual  velocity  of  the  robot.  Figure  1  shows 
a  typical  backprojection  of  a  disk  of  radius  r.  The  commanded  velocity  in  this  case  is  v,  which 
points  in  the  direction  (  —  1,0),  and  the  velocity  error  ball  has  radius  ev  |v|,  with  tv  =  For 
each  possible  velocity  direction  one  can  compute  such  a  backprojection. 

These  backprojections  are  larger  than  what  we  want.  Recall  that  our  basic  model  of  an 
action  is  a  pair:  a  velocity  v0  and  a  time  At.  This  model  of  action  leads  to  time-indexed 
backprojections.  See  Figure  2.  The  semantics  are  as  follows.  If  the  robot  starts  off  in  the 
crescent- shaped  region  depicted  in  the  figure,  then  upon  execution  of  velocity  v0  for  time 
At ,  the  robot  is  guaranteed  to  be  somewhere  inside  the  disk  of  radius  r*,  despite  control 
uncertainty.  Formally,  we  have  that  the  time-indexed  backprojection  of  a  goal  Q  under  the 
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Figure  2:  This  figure  shows  a  typical  time-indexed  backprojection  of  a  two-dimensional  disk.  The 
goal  disk  has  radius  r*  and  is  centered  at  the  origin.  The  action  is  to  execute  the  nominal  velocity 
v0  for  time  At.  The  velocity  uncertainty  is  given  by  an  error  ball  with  radius  c„  |vo|.  The  resulting 
time-indexed  backprojection  is  a  disk  with  radius  qk  =  rk  -  €v  |v0|  At  centered  at  (a,0)  =  -v0  At. 
The  shaded  crescent-shaped  area  is  the  portion  of  the  backprojection  that  lies  outside  of  the  goal 
disk.  The  point  p*  is  one  of  the  intersection  points  of  the  circles  that  circumscribe  the  two  disks. 
For  comparison,  the  dashed  lines  outline  the  full  backprojection.  See  also  Figure  1. 

action  (v0,  At)  is  the  set  $Vo,At(£)?  with 

BvoMG)  =  {p  |  fle.|v0|At(p  +  v<>Af)  eg}. 

Here  p  is  a  point  in  the  plane  and  BCv  jVo|  a«(p  +  v0  At)  is  the  two-dimensional  ball  of  radius 
tv  |v0|  At  centered  at  the  position  p  +  v0  At.  (We  are  assuming  no  obstacles.) 

Now,  observe  that  if  Q  is  a  disk  of  radius  r  centered  at  the  origin,  then  we  can  rewrite 
By 0,At(G)  as 

By0,At(Q)  =  {p  |  |p  4-  v0  At  +  Ap|  <  r,  for  every  Ap  with  |Ap|  <  ev  |v0|  At} 

=  {p  |  |p  + v0Atj  <  r -e„|v0j  At} 

Br— tv  |vo|  At(  Vo  At). 

In  other  words,  the  time-indexed  backprojection  of  a  disk  of  radius  r  is  formed  by  first 
shrinking  the  radius  of  this  disk  by  the  positioning  uncertainty  e„  |vo|  At,  then  shifting  the 
resulting  disk  by  the  negative  of  the  commanded  positional  change  v0  At.  This  is  exactly  what 
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one  would  expect  intuitively.  Figure  2  shows  both  a  goal  disk  of  radius  and  its  time-indexed 
backprojection. 

5.4  Planning  A  Strategy 

During  the  first  of  the  backchaining  stages,  the  goal  disk  in  Figure  2  corresponds  to  the  task- 
level  goal.  In  later  stages,  the  disk  constitutes  a  composite  subgoal,  formed  from  the  union  of 
several  crescent-shaped  backprojections. 

During  a  given  stage  of  the  backchaining  process,  the  planner  expands  the  current 
composite  subgoal  of  radius  rk  to  a  new  composite  subgoal  of  radius  +  (1  —  ev)  |v0|  At. 
Specifically,  the  planner  computes  time-indexed  backprojections  for  all  possible  velocity 
directions.  It  holds  fixed  the  magnitude  of  the  commanded  velocity  |vo|  and  the  duration 
of  time  At ,  but  rotates  the  direction  of  the  commanded  velocity  through  2 tt  radians.  The 
union  of  the  resulting  crescent-shaped  backprojections  forms  a  band  of  width  (1  —  e„)  |v0|  At 
about  the  old  composite  subgoal  of  radius  rk.  Taken  together,  the  band  and  the  old  subgoal 
form  the  new  subgoal. 

The  idea  is  that  if  the  robot  starts  off  in  a  crescent-shaped  backprojection,  and  if  the 
executive  commands  velocity  v0  for  time  At,  then  the  robot  is  guaranteed  to  wind  up  in  the 
composite  subgoal  disk.  Once  in  this  disk,  the  robot  can  then  execute  whatever  strategy  it 
has  recursively  computed  for  attaining  the  next  composite  subgoal,  and  so  forth. 

5.5  Differential  Backchaining 

Suppose  that  one  repeatedly  computes  time-indexed  backprojections,  starting  with  the  goal 
disk  of  radius  r,  and  backchaining  over  and  over.  At  the  k  +  lat  stage  one  has  a  composite 
subgoal  of  radius  rk  centered  at  the  origin.  Here  k  runs  from  0  on  upward,  with  r0  =  r. 
Given  rk  and  v0  At,  one  computes  a  time-indexed  backprojection,  which  is  a  disk  of  radius 
qk  centered  at  — Vo  At.  The  circles  bounding  these  two  disks  intersect  at  the  point  p*  and  its 
mirror  image  (relative  to  the  line  given  by  v0). 

For  simplicity,  let  us  assume  that  v0  is  parallel  to  the  x-axis  and  that  it  is  pointing  in  the 
negative  x  direction.  Let  us  define  a  =  |vo  At\,  and  write  p*  =  ( Xk,yk )•  Then  we  observe  that 
qk  =  rk  -  ev  a  and  that  rk  =  r  +  k  (1  —  e„)  a.  By  solving  for  the  intersection  of  two  circles,  it 
follows  that 


xk  =  rev  +  k(  1  -  ev)eva  +  i(l  -  e2v)a. 

Furthermore,  yk  is  implicitly  defined  by  the  equation  x\  +  y\  =  r\. 

The  interesting  question  is  what  happens  as  we  make  the  time  constant  At  of  the  feedback 
loop  very  small.  Suppose  we  let  a  — ►  0.  In  order  to  backchain  out  to  some  location  p,  we 
must  therefore  let  k  approach  infinity  in  such  a  way  that  the  product  ka  remains  constant, 
say  at  some  value  c  that  depends  on  p.  Thus  we  have  that 


Xk  -*  (r  +  c(l  -  e„)), 
yk  -*  y/l  -€*  (r  +  c(l  -e„)). 
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Figure  3:  The  two  solid  lines  bound  a  cone  of  progress,  which  is  indicated  by  the  shaded  region. 
The  nominal  commanded  velocity  v0  is  the  vector  (—1,0),  with  uncertainty  e„  =  0.25.  If  the  robot 
starts  within  the  cone  of  progress,  then  commanding  velocity  v0  for  a  differential  amount  of  time 
is  guaranteed  to  move  the  robot  closer  to  the  origin,  despite  the  uncertainty  in  control.  Each  pair 
of  circles  comprises  a  composite  subgoal  disk  and  its  time-indexed  backprojection  relative  to  the 
velocity  v0  =  (—1,0)  and  the  time  duration  At  =  0.1.  The  radius  of  the  first  subgoad  is  r0  =  1. 
Observe  that  the  solid  lines  nearly  trace  out  the  intersection  points  of  each  subgoal  disk  with  its 
time-indexed  backprojection.  As  At  becomes  smaller,  this  match  becomes  better.  See  also  Figure  2. 

We  therefore  see  that 


Vk_  \A  ~  el 

Xk  ev 

In  other  words,  the  intersection  points  {p*}  line  up  in  a  straight  line  with  slope  yj\  —  t*/ e„. 

One  way  to  interpret  this  is  as  follows.  Suppose  we  perform  the  backchaining  process  for 
all  velocities  and  all  execution  times,  until  we  have  covered  the  ’.tire  plane  with  composite 
subgoals.  Now,  fix  the  velocity  vector  v0  =  (—1,0),  and  allow  the  time  At  to  approach  zero. 
Consider  the  union  of  all  the  crescent-shaped  portions  of  the  time-indexed  backprojections 
computed  during  the  backchaining  process  relative  to  the  actions  {(v0,  Af)}^uo-  The  union  is 
a  semi-infinite  cone,  with  apex  at  the  origin.  The  cone  is  bounded  by  two  lines  whose  slopes 
are  ±^1  —  j tv.  Figure  3  depicts  the  bounding  cone.  Also  shown  in  the  figure  are  pairs  of 
circles.  In  each  case  one  circle  represents  a  composite  subgoal  of  radius  r*  for  some  k,  while 
the  other  circle  has  radius  qk  and  is  a  time-indexed  backprojection  of  the  subgoal,  for  a  small 
value  of  At. 

What  does  this  cone  mean?  The  interior  of  the  cone  consists  of  all  those  locations  at  which 
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the  nominal  velocity  v0  is  guaranteed  to  make  progress  towards  the  goal  for  a  differential 
amount  of  time.  This  is  clear  from  the  manner  in  which  the  cone  was  constructed.  It  is  also 
clear  when  one  observes  that  each  of  the  bounding  edges  of  this  cone  is  perpendicular  to  one 
of  the  extreme  velocity  vectors  in  the  velocity  error  ball  about  the  nominal  velocity  v0.  This 
means  that  if  the  robot  is  located  on  the  edge  of  the  cone  then  there  is  a  possible  velocity 
which  will  cause  the  robot  to  remain  differentially  at  the  same  distance  from  the  goal. 


5.6  Backchained  Actions  Provide  an  Implicit  Progress  Measure 

It  is  useful  to  understand  the  reason  that  backchaining  generates  such  cones.  Effectively,  the 
backchaining  process  defines  an  implicit  progress  measure  on  the  plane.  For  instance,  consider 
the  case  in  which  we  hold  |vo|  A t  constant,  while  varying  the  direction  of  v0.  In  that  case, 
the  number  of  backchaining  steps  k  required  in  order  to  include  a  given  point  p  of  the  plane 
within  some  subgoal  disk  of  radius  rk  defines  a  discrete  progress  measure.  In  the  general  case, 
as  k  — ►  oo  and  At  — ►  0,  we  see  that  for  a  given  point  p  the  product  k  (1  —  e„)  a  simply  reduces 
to  the  distance  of  the  point  from  the  task  goal. 

With  this  description  in  mind,  we  will  refer  to  the  cone  as  a  cone  of  progress.  Specifically, 
it  is  a  cone  of  progress  relative  to  the  commanded  velocity  v0. 

5.7  Cones  of  Progress 

The  previous  construction  should  come  as  no  surprise.  The  cone  appeared  in  a  different  form 
in  [Erdmann  1990]  in  the  context  of  randomization.  That  work  discusses  a  simple  feedback 
loop  for  accomplishing  the  point-into-disk  task,  despite  uncertainty  in  sensing  and  control. 
The  feedback  loop  operated  as  follows.  The  robot  would  sense  its  current  position.  If  all 
position  interpretations  consistent  with  the  observed  sensor  value  lay  wholly  within  some  cone 
of  progress,  then  the  robot  would  determine  a  velocity  vector  and  a  maximum  execution  time 
that  would  permit  the  robot  to  move  closer  to  the  goal.  On  the  other  hand,  if  no  single  cone 
of  progress  contained  all  the  possible  locations  of  the  robot,  then  no  action  was  guaranteed  to 
move  the  robot  closer  to  the  goal.  In  that  case,  the  robot  would  execute  a  random  motion. 

There  are  two  important  ideas  contained  in  the  last  paragraph.  The  first  concerns  speed  of 
progress.  The  second  is  a  description  of  a  better  sensor.  First,  in  the  randomization  example 
of  [Erdmann  1990],  the  sensor  was  a  cartesian  sensor  whose  uncertainty  was  modelled  as  an 
error  ball.  This  meant  that  for  observed  sensor  values  close  to  the  origin,  the  system  could 
not  decide  that  its  position  lay  in  any  particular  cone  of  progress.  Conversely,  for  observed 
sensor  values  far  away  from  the  origin,  the  location  of  the  sensor  interpretation  set  inside  a 
cone  of  progress  determined  the  amount  of  progress  that  the  system  could  make.  In  other 
words,  we  see  that  it  is  the  relationship  of  a  particular  sensor  and  sensed  value  to  all  the  cones 
of  progress  that  determines  whether  progress  is  possible  and  if  so,  hew  much  progress.  This 
is  the  sensing-speed  tradeoff  that  we  mentioned  earlier. 

The  second  idea  concerns  the  design  of  a  task-specific  sensor.  We  argued  earlier  based  on 
simple  symmetry  that  the  sensor  should  be  an  angular  sensor  that  is  centered  at  the  goad  and 
that  looks  out  from  the  goal  toward  the  robot.  That  argument  finds  further  support  in  the 
current  geometric  reasoning.  In  order  for  the  robot  to  be  certain  that  a  particular  action  will 
move  it  closer  to  the  goal,  the  robot  needs  to  know  that  its  current  position  lies  within  the 
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cone  of  progress  associated  with  that  action.  This  requirement  points  towards  the  design  of 
a  sensor  that  recognizes  cones  of  progress.  For  the  example  above,  this  means  that  we  need 
a  sensor  that  can  decide  whether  the  robot  is  located  in  some  conical  region  with  half-angle 
sin-1(e„)  that  is  centered  at  the  goal. 

In  summary:  Good  sensors  should  recognize  cones  of  progress. 

A  corollary:  Sensors  should  not  recognize  states; 

Sensors  should  recognize  applicable  actions. 

5.8  Coverings  by  Cones  of  Progress 

There  axe  two  technicalities  that  we  have  yet  to  address.  First,  designing  a  sensor  to  recognize 
a  single  cone  of  progress  is  not  enough.  The  entire  state  space  need  not  be  covered  by  a  single 
cone  of  progress.  This  leads  us  to  a  family  of  sensors.  At  one  extreme  we  have  a  sensor  that  is 
capable  of  recognizing  the  robot’s  presence  or  absence  in  each  and  every  cone  of  progress.  For 
the  example  above  this  amounts  to  a  perfect  angular  sensor.  Given  such  a  sensor  we  would 
at  run-time  see  the  robot  executing  a  nearly  straight-line  trajectory  from  its  initial  location 
to  the  goal. 

At  the  other  extreme  we  have  a  sensor  that  is  capable  at  run-time  of  recognizing  the  robot’s 
presence  in  at  least  one  cone  of  progress,  but  not  necessarily  more.  One  view  of  such  a  sensor 
is  as  follows.  We,  as  sensor  designers,  decide  on  a  covering  of  the  state  space  by  some  small 
collection  of  cones  of  progress.  We  then  design  a  sensor  that  can  recognize  this  covering.  In 
other  words,  at  run-time  the  sensor  can  accurately  report  the  robot’s  position  in  at  least  one 
cone  of  progress.  There  may  be  many  such  sensors,  since  there  may  be  many  different  minimal 
coverings  of  the  state  space. 

As  an  example,  for  small  values  of  c„,  the  state  space  of  the  point-into-disk  task  can  be 
covered  by  four  cones  of  progress.  For  instance,  we  might  take  these  four  cones  to  be  the 
cones  of  progress  whose  axes  are  aligned  with  the  four  half-axes  of  the  x-y  coordinate  system. 
Given  this  covering,  the  sensor  merely  has  to  decide  in  which  of  four  angular  regions  the  robot 
is  located.  At  run-time  we  would  observe  the  robot  executing  a  series  of  velocities  whose 
commanded  directions  are  aligned  with  the  axes.  The  net  effect  would  be  to  pull  the  robot  in 
towards  the  goal  along  a  path  near  the  diagonals  of  the  state  space.  The  speed  of  progress  in 
this  case  is  less  than  that  achievable  with  a  perfect  angular  sensor. 

5.9  Noise 

The  second  technicality  that  we  need  to  address  is  noise.  Consider  a  particular  physical 
implementation  of  a  sensor.  We  can  view  the  sensor  as  mapping  an  observed  sensed  value  to 
a  collection  of  cones  of  progress,  each  of  which  is  certain  to  contain  the  actual  position  of  the 
robot.  A  perfect  sensor  maps  an  observed  sensed  value  to  all  cones  of  progress  that  contain  the 
robot’s  actual  location.  As  the  sensor  becomes  noisier,  the  sensor  maps  the  observed  sensed 
value  to  a  smaller  collection  of  cones  of  progress. 

So  long  as  the  mapping  from  observed  sensor  values  to  collections  of  cones  of  progress  is 
well-defined,  there  is  no  problem.  Progress  may  be  slow,  but  so  long  as  there  is  at  least  one 
cone  of  progress  for  each  possible  sensor  value,  progress  is  certain.  In  the  case  of  the  cartesian 
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sensor  mentioned  earlier,  we  saw  that  the  mapping  was  not  always  well-defined.  In  particular, 
for  observed  robot  positions  close  to  the  goal,  there  was  no  cone  of  progress  that  was  certain 
to  contain  the  actual  position  of  the  robot. 

For  the  point-into-disk  task  above,  we  can  imagine  a  physical  sensor  that  returns  some 
angle  9*  as  an  estimate  of  the  robot’s  position  relative  to  the  goal.  To  say  that  the  sensor  is 
perfect  is  to  say  that  the  robot  is  located  precisely  at  that  orientation  relative  to  the  goal.  To 
say  that  the  sensor  is  noisy  may  mean  that  the  actual  angular  location  of  the  robot  lies  in 
some  interval  (9*  —  7,  #*  +  7),  where  7  is  an  error  angle  that  measures  the  sensor’s  uncertainty. 
So  long  as  7  is  less  than  sin_1(e„),  we  can  be  sure  that  the  sensor  can  recognize  a  cone  of 
progress.  This  condition  thus  creates  a  design  constraint  on  any  physical  implementation  of 
the  abstract  sensor  defined  by  the  cones  of  progress. 

5.10  Summary 

The  cones  of  progress  specify  exactly  what  knowledge  the  robot  must  have  in  order  to  make 
progress,  and  how  the  available  information  affects  the  speed  of  progress.  A  sensor  should  be 
viewed  as  recognizing  cones  of  progress.  Given  this  description,  one  can  then  design  physical 
sensors  that  try  to  recognize  as  many  cones  of  progress  as  one  requires  in  order  to  accomplish 
a  task  quickly. 

6  Ongoing  and  Proposed  Research 

6.1  Two-Pin  Two-Hole  Task 

We  axe  currently  studying  the  problem  of  inserting  a  two-pin  part  into  a  pair  of  circular 
holes.  In  particular,  we  are  examining  the  character  of  a  robust  sensor  that  provides  precisely 
the  information  needed  to  perform  this  insertion  as  quickly  as  possible.  The  problem  forces  us 
to  consider  rotations  in  addition  to  translations.  We  are  studying  this  problem  from  several 
perspectives.  In  particular,  we  have  performed  a  theoretical  analysis,  we  have  written  a 
simulator,  and  we  have  programmed  the  task  as  an  assembly  operation  on  the  Zebra  Zero 
manipulator.  The  theoretical  considerations  suggest  that  the  ideal  sensor  is  a  sensor  that 
recognizes  radial  lines  in  the  ( x,y,9 )  configuration-space  of  the  two-pin  part.  It  is  difficult  to 
implement  such  a  sensor  physically.  However,  by  combining  two  of  the  translational  radial 
sensors  used  for  the  single  peg-into-hole  problem,  one  can  obtain  an  approximation  to  the 
ideal  sensor.  See  Figure  4.  This  approximation  provides  a  close  match  to  the  ideal  sensor  for 
configurations  of  the  part  near  the  goal,  where  the  sensor  is  needed  the  most.  In  particular, 
the  approximation  improves  as  the  part  moves  towards  its  goal,  obtaining  an  ideal  match  at 
the  goal  configuration.  Such  limiting  behavior  is  unusual,  in  that  standard  cartesian  sensors, 
that  are  not  explicitly  designed  for  a  specific  task,  exhibit  growing  divergence  as  the  part 
moves  towards  the  goal. 

We  expect  that  for  many  tasks  the  design  of  a  task-specific  sensor  will  involve  steps 
similar  to  those  for  the  two-pin  part.  The  design  process  will  use  a  configuration-space 
analysis  coupled  with  a  backchaining  planner  to  generate  an  ideal  sensor.  The  resulting 
configuration-space  sensor  will  provide  a  description  of  the  precise  information  required  to 
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Figure  4:  Relationship  of  two  radial  sensors  to  the  configuration  of  a  two-pin  part.  Each  sensor  is 
centered  at  one  of  the  goal  holes.  The  sensors  report  back  two  angles,  (#i,#2)-  is  the  angle  that 
pin  i  makes  relative  to  hole  t. 

solve  the  task.  The  ideal  sensor  will  generally  have  no  immediate  physical  realization.  Instead, 
the  sensor  serves  as  a  guide,  suggesting  different  physical  implementations  as  approximations. 
An  important  and  as  yet  unexplored  step  of  the  design  process  is  the  transformation  of  an 
ideal  configuration-space  sensor  into  a  physically  valid  sensor. 

6.2  Theoretical  Issues 

There  are  numerous  interesting  theoretical  questions  that  we  would  like  to  explore: 

•  The  construction  of  Section  5  is  almost  a  general  algorithm.  We  would  like  to  describe 
the  general  process  for  designing  an  ideal  sensor.  We  believe  that  in  many  cases  this 
general  process  is  of  the  same  character  as  the  preimage  procedures  for  dealing  with 
uncertainty.  Part  of  the  motivation  for  our  research  is  the  belief  that  this  process  is  in 
many  practical  cases  an  algorithm,  and  that  it  is  an  algorithm  of  low  complexity.  This 
belief  needs  to  be  tested. 
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•  We  have  described  the  ideal  sensor  in  this  proposal  as  a  physical  device  that  supplies 
information  on  demand.  This  description  is  an  oversimplification.  The  required 
information  cam  arise  in  many  different  ways;  the  information  need  not  be  the  direct 
output  of  a  physical  sensor.  For  instance,  a  robot  might  obtain  the  information  it  requires 
to  make  progress  by  performing  information-gathering  motions  and  by  maintaining  a 
history  of  past  motions  and  sensor  values.  The  research  question  is  whether  it  is  possible 
to  combine  history  and  prediction  with  a  tight  sensory  feedback  loop  of  the  type  that 
we  are  assuming. 

As  an  example,  in  a  particular  case  of  the  peg-in-hole  problem,  it  may  be  necessary  for  a 
robot  to  use  a  cartesian  sensor  with  a  Kalman  filter,  instead  of  the  ideal  radial  sensor  that 
falls  out  of  the  design  process.  As  the  peg  approaches  close  to  the  hole,  the  sensor  may 
suddenly  not  provide  the  information  required  to  make  further  progress.  In  particular, 
if  ever  the  sensing  error  overlaps  several  cones  of  progress,  then  progress  may  not  be 
possible.  Fortunately,  the  cones  of  progress  also  provide  a  description  of  the  amount 
of  information  required  to  again  make  progress.  Thus,  one  can  compute  the  expected 
amount  of  time  that  the  robot  must  simply  sit  and  repeatedly  consult  its  sensor.  While 
sitting,  the  robot  uses  the  Kalman  filter  to  reduce  the  variance  in  its  position  estimate, 
until  the  sensor  error  lies  wholly  within  one  cone  of  progress.  Similarly,  once  the  peg 
overlaps  the  hole,  the  robot  could  use  torque  sensing  to  obtain  the  information  specified 
by  the  cones  of  progress. 

The  research  question  is  whether  one  can  efficiently  automate  the  construction  of 
information-gathering  subtasks  for  robots  that  are  not  equipped  with  ideal  sensors. 

•  One  of  the  primary  motivations  of  the  proposed  research  is  to  relax  the  assumption  that 
sensors  are  fixed  and  given  devices.  Instead,  a  system  should  design  sensors  to  suit  its 
available  actions  and  its  current  task.  An  extension  of  this  idea  is  to  mechanically  and 
physically  modify  the  environment  of  the  task,  so  that  a  given  action  will  make  progress 
towards  the  goal  from  a  larger  set  of  initial  conditions  them  is  first  possible.  The  cones 
of  progress  specify  tlje  extent  to  which  a  sensor  must  distinguish  between  actions.  The 
cones  of  progress  thus  also  specify  which  actions  could  be  merged  by  redesigning  the 
mechanics  of  the  task. 

•  In  designing  approximations  to  ideal  configuration  space  sensors  it  is  useful  to  have  a 
small  repertoire  of  primitive  sensors,  along  with  a  calculus  for  combining  these  sensors 
to  build  more  complicated  sensors.  We  are  interested  in  how  simple  sensors  may  be 
combined  into  more  powerful  sensors.  A  promising  approach  is  to  use  a  number  of 
movable  light-beam  sensors  that  can  measure  distance  very  accurately  in  a  small  local 
volume.  We  are  motivated  here  by  the  recent  successes  of  [Canny  and  Goldberg  1991]. 


6.3  Proposed  Experimental  Domain 

We  propose  to  test  our  sensor  design  methodology  in  the  domain  of  parts  assembly.  We 
will  consider  planar  parts  assembly  and  planar  parts  orienting.  In  addition  we  will  consider 
three-dimensional  assembly  operations  in  which  the  essential  character  of  the  assembly  can 
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be  represented  in  low  dimensions,  such  as  the  cylindrical  peg-in-hole  and  the  two-pin-two-hole 
problems  discussed  above. 

The  types  of  tasks  we  envision  are: 

•  Locating  parts  that  enter  the  workarea  of  the  robot  via  some  transport  mechanism,  such 
as  a  feeder  belt.  The  parts  that  we  have  in  mind  are: 

—  Rigid  planar  parts,  of  different  sizes  and  shapes.  Some  of  these  shapes  are  convex, 
others  concave.  Some  contain  holes  for  attaching  springs  or  bolts. 

—  Blocks  of  varying  sizes. 

—  Bolts  of  varying  sizes,  shapes,  and  thread  types. 

—  Nuts  corresponding  to  the  bolts. 

-  Springs  of  varying  lengths. 

•  Distinguishing  between  different  parts  that  enter  the  workarea. 

•  Orienting  and  positioning  the  parts. 

•  Assembling  the  parts.  The  robot  most  accomplish  several  different  kinds  of  assembly 
tasks,  of  the  form: 

—  Assemble  two  or  more  of  the  planar  shapes,  that  are  designed  to  fit  together. 

-  Fasten  two  or  more  parts  together  with  a  bolt. 

-  Fasten  two  or  more  parts  together  with  a  spring. 

The  physical  hardware  that  we  would  like  to  employ  is  a  Zebra  Zero  force-controlled 
manipulator,  a  vision  system,  and  a  number  of  very  simple  sensors  such  as  light  beams, 
infrared  proximity  sensors,  and  contact  switches.  We  currently  have  the  Zebra  manipulator 
running  in  the  lab.  We  would  like  to  purchase  a  vision  system,  the  simple  sensors,  and  better 
workstations  on  which  to  run  our  control  and  planning  software.  In  particular,  the  Zebra 
may  be  controlled  either  directly  from  a  PC  or  over  the  network  from  a  Sun  workstation. 
The  PC  does  not  permit  us  to  run  very  extensive  planning  systems,  while  the  Sun  is  an  old 
3/60  model  with  very  little  memory  and  a  slow  processor,  resulting  in  significant  paging  and 
network  delays.  We  would  like  to  upgrade  our  hardware  in  order  to  perform  efficiently  the 
experimental  part  of  our  research. 

The  main  objective  of  the  experimental  part  of  the  proposed  research  is  to  test  the 
versatility  of  an  assembly  system  that  can  design  its  own  sensors.  We  intend  to  write 
planning  software  to  design  the  sensors.  We  envision  a  planner  that  expects  as  input  a 
geometric  description  of  the  parts  to  be  assembled,  that  has  some  knowledge  of  mechanics  and 
uncertainty,  and  that  can  reason  about  combining  primitive  sensors  to  form  more  complicated 
sensors.  Additionally,  the  planner  will  expect  as  input  a  geometric  description  of  an  assembly 
task.  The  planner  will  backchain  from  the  goal  of  this  task,  in  order  to  generate  both  a  desired 
sequence  of  assembly  operations  that  accomplishes  the  task  as  well  as  an  ideal  sensor  that 
provides  the  information  required  to  execute  the  assembly  operations.  Having  found  such  a 
plan,  the  planner  will  then  approximate  the  ideal  sensor  with  a  combination  of  the  primitive 
sensors. 
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7  A  History  of  Manipulation  Methodologies 

7.1  Symbolic  Planning 

The  SHAKEY  system  from  the  late  sixties  (see  [Nilsson  1980,  1984]  for  descriptions)  is  a 
splendid  example  of  an  early  AI  approach  for  planning  the  motions  of  a  robot  operating  in 
a  physical  world.  The  system  consisted  of  a  mobile  robot,  a  high-level  planner  based  on 
STRIPS,  and  several  layers  of  lower-level  planning  and  control  loops.  The  goal  set  forth  by  the 
SHAKEY  system  has  driven  research  in  robotics  and  AI  for  several  decades.  The  goal  was  to 
develop  a  system  that  would  permit  specification  of  robot  goals  at  the  task  level.  The  system 
was  expected  to  then  translate  these  task-level  goals  into  appropriate  commands  for  actually 
moving  the  robot  and  accomplishing  the  task. 

Two  difficulties  became  apparent  with  SHAKEY.  The  first  is  that  uncertainty  mattered. 
Someone  had  to  implement  low-level  routines  to  deal  with  uncertainty.  The  second  difficulty 
is  that  the  symbolic  states  used  to  represent  the  robot’s  configuration  were  too  simple.  Once 
one  proceeds  to  more  complicated  tasks,  such  as  the  assembly  of  complicated  parts,  these 
symbolic  states  are  insufficient.  Instead,  it  seems  to  be  necessary  to  consider  geometry  in 
detail. 

These  two  difficulties  led  to  further  work.  Some  of  the  work  dealt  with  uncertainty,  other 
work  dealt  with  geometry.  Later  these  two  threads  reconnected  to  form  a  general  approach 
for  dealing  with  uncertainty  in  geometrically  specified  environments. 

7.2  Configuration  Space 

The  work  on  geometry  sought  to  understand  the  constraints  imposed  by  obstacles  in  the  world 
on  the  motions  of  a  robot  or  manipulator.  In  a  series  of  articles,  Lozano-Perez  developed  the 
notion  of  configuration  space  as  a  means  of  characterizing  these  constraints.  See  [Lozano-Perez 
and  Wesley  1979]  and  [Lozano-Perez  1981,  1983]. 

The  configuration  space  approach  spawned  more  than  a  decade  of  work  on  motion  planning. 
Some  work  sought  to  classify  the  complexity  of  the  motion  planning  problem  under  the 
assumptions  made  above,  and  to  devise  and  implement  algorithms  for  planning  solutions 
to  this  problem.  See,  for  instance,  [Reif  1979],  [Schwartz  and  Sharir  1983a],  [Donald  1984, 
1987],  [Canny  1986,  1988a,  1988b],  [Khatib  1986],  and  [Koditschek  1987].  Ultimately  the 
problem  was  classified  as  PSPACE-complete  (see  [Reif  1979]  and  [Canny  1988b]).  Other 
work  considered  the  problem  of  planning  the  motions  for  several  cooperating  robots.  See,  for 
instance,  [Schwartz  and  Sharir  1983b],  [Yap  1984],  [Hopcroft,  Schwartz,  and  Sharir  1984],  [Reif 
and  Sharir  1985],  [Kant  and  Zucker  19S6],  and  [Erdmann  and  Lozano-Perez  1987].  Some  books 
with  excellent  survey  articles  include  [Brady  et  al.  1982],  [Schwartz  and  Yap  1986],  [Schwartz, 
Hopcroft,  and  Sharir  1987],  [Khatib,  Craig,  and  Lozano-Perez  1989],  and  [Latombe  1990]. 

7.3  Guarded  Moves  and  Compliant  Motion 

Early  work  on  uncertainty  sought  to  improve  the  performance  of  robot  planning  and  execution 
systems  by  developing  primitive  operations  specifically  designed  to  overcome  uncertainty.  The 
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intent  was  to  retain  the  structure  of  high-level  planning  systems,  while  using  the  primitive 
operations  to  encapsulate  the  low-level  activity  needed  to  overcome  uncertainty. 

The  new  primitives  were  special  feedback  loops  and  force  control  strategies  that  coupled 
motion  to  sensing  in  a  manner  that  reduced  uncertainty.  Guarded  moves  consisted  of  motions 
that  terminated  execution  when  some  sensory  condition  was  satisfied.  Similarly,  compliant 
motion  strategies  were  developed  to  maintain  contact  with  surfaces.  These  strategies  were 
useful  for  establishing  and  maintaining  contact  with  a  surface  even  when  the  location,  shape, 
or  attitude  of  the  surface  was  uncertain.  See  [Ernst  1961],  [Will  and  Grossman  1975],  [Drake 
1977],  [Whitney  1977],  [Salisbury  1980],  [Mason  1981],  and  [Raibert  and  Craig  1981],  among 
others. 

Related  activity  involved  the  construction  of  mechanical  devices,  such  as  the  RCC  of 
Draper  Labs  that  could  perform  tight  assembly  operations  without  requiring  precise  control 
or  sensing.  The  construction  of  such  devices  required  a  physical  analysis  of  the  task  that 
involved  modelling  both  geometry  and  friction.  See  [Nevins  et  al.  1975].  Indeed,  this  work 
motivated  other  work  on  modelling  friction  and  on  planning  mechanical  operations  guaranteed 
to  succeed  in  the  presence  of  uncertainty  (see,  for  instance,  [Simunovic  1979],  [Whitney  1982], 
[Erdmann  1984],  [Caine  1985],  and  [Sturges  1988]). 

7.4  Skeleton  Strategies 

Guarded  moves  and  compliant  motions  provided  local  tools  for  overcoming  uncertainty.  The 
natural  next  step  was  to  incorporate  these  local  strategies  into  the  global  motion  planners. 

The  first  step  was  to  relax  the  view  of  assembly  strategies  as  fixed  loops  based  on  nominal 
robot  and  part  trajectories.  Early  work  considered  parameterizing  strategies  in  terms  of 
quantities  that  could  vary  with  particular  problem  instantiations.  The  skeleton  strategies  of 
[Lozano-Perez  1976]  and  [Taylor  1976]  offered  a  means  of  relating  error  estimates  to  strategy 
specifications  in  detail.  [Brooks  1982]  extended  this  approach  using  a  symbolic  algebra  system. 
His  system  could  be  used  both  to  provide  error  estimates  for  given  operations,  as  well  as 
to  constrain  task  variables  or  add  sensing  operations  in  order  to  guarantee  task  success. 
Along  a  slightly  different  line,  [Dufay  and  Latombe  1984]  developed  a  system  that  observed 
execution  traces  of  proposed  plans,  then  modified  these  using  inductive  learning  to  account 
for  uncertainty. 

7.5  Preimages 

In  the  previous  research  we  see  the  first  attempts  to  account  for  uncertainty.  However, 
uncertainty  is  still  treated  merely  as  an  irritant,  capable  of  destroying  a  previously  and 
independently  developed  plan.  In  1983,  Lozano-Perez,  Mason,  and  Taylor  proposed  a 
planning  framework  for  synthesizing  fine-motion  strategies  in  the  presence  of  uncertainty. 
This  framework  generates  plans  by  recursively  backchaining  from  the  goal.  Each  backchaining 
step  generates  a  collection  of  sets,  known  as  preimages ,  from  which  entry  into  the  goal 
is  guaranteed,  despite  sensing  and  control  uncertainty.  The  preimage  framework  directly 
incorporates  uncertainty  into  the  planning  process.  See  [Mason  1984],  [Erdmann  1986], 
[Buckley  1987],  [Donald  1989],  and  [Canny  1989]  for  further  work  on  preimages.  See  [Latombe 
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1990]  for  an  excellent  discussion  of  preimage  work,  and  see  [Bertsekas  1987]  for  related  work 
in  the  stochastic  domain. 


7.6  Sensor  less  Manipulation 

Open-loop  strategies  constitute  an  important  class  of  manipulation  strategies,  some  of  which 
axe  explicitly  designed  to  reduce  uncertainty.  Such  sensorless  strategies  rely  purely  on  the 
mechanics  of  their  domain  to  accomplish  a  task,  despite  uncertainty  in  the  actions  and  in  the 
initial  state  of  the  system.  See,  for  instance,  [Mason  1982,  1986].  In  particular,  Mason  looked 
at  the  problem  of  reducing  uncertainty  in  the  orientation  of  parts  by  pushing.  Building 
on  this  work,  [Brost  1988]  implemented  a  system  that  could  orient  planar  parts  through  a 
series  of  pushing  and  squeezing  operations.  See  also  [Mani  and  Wilson  1985],  [Peshkin  1986], 
[Natarajan  1986],  [Erdmann  and  Mason  1988],  and  [Goldberg,  Mason,  and  Erdmann  1991]. 

7.7  Error  Detection  and  Recovery 

An  important  offspring  of  the  LMT  preimage  planning  methodology  is  Donald’s  work  on 
Error  Detection  and  Recovery  [Donald  1988a,  1989,  1990].  This  work  deals  with  the  related 
problems  of  representing  model  error  and  planning  strategies  for  error  detection  and  recovery. 
Donald’s  work  moved  away  from  the  requirement  that  a  strategy,  in  order  to  be  considered  a 
legitimate  strategy,  actually  be  guaranteed  to  solve  a  task  in  a  fixed  predetermined  number 
of  steps.  This  move  represents  an  important  change  in  perspective. 


7.8  Randomization 

Building  on  top  of  Donald’s  ideas,  [Erdmann  1990]  studied  active  randomization  as  a  primitive 
strategy  for  accomplishing  robot  tasks.  Randomization  is  one  way  to  overcome  the  problem 
posed  by  Donald.  This  is  the  problem  of  deciding  what  to  do  if  a  system  cannot  guarantee 
task  success  with  the  actions  and  sensors  that  it  has  available.  Randomization  accomplishes  a 
task  probabilistically.  By  actively  randomizing  its  actions,  a  system  can  blur  the  significance 
of  unmodelled  or  uncertain  parameters. 

An  interesting  consequence  of  this  view  is  that  there  are  tasks  that  may  be  solved  with  no 
information  except  for  a  sensor  that  signals  goal  attainment.  For  instance,  for  tasks  in  which 
there  is  no  danger  of  catastrophic  failure  and  in  which  there  are  no  trap  states,  a  robot  can 
simply  perform  a  random  walk  through  state  space  until  the  goal  is  attained. 

Randomization  has  been  used  explicitly  for  solving  automation  tasks  by  other  researchers. 
In  the  domain  of  mobile  robots,  see  for  instance  [Arkin  1989],  who  injects  noise  into  potential 
fields  to  avoid  plateaus  and  ridges.  [Barraquand  and  Latombe  1989]  investigated  a  Monte- 
Carlo  approach  for  escaping  from  local  minima  in  potential  fields.  Some  probabilistic  work  was 
aimed  at  facilitating  the  design  process.  For  instance,  [Boothroyd  et  al.  1972]  considered  the 
problem  of  determining  the  natural  resting  distributions  of  parts  in  a  vibratory  bowl  feeder. 
[Goldberg  1990]  investigated  probabilistic  strategies  for  grasping  objects,  and  developed  a 
framework  for  planning  optimal  orienting  strategies  relative  to  various  cost  functions. 
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7.9  Interwoven  Strategies  and  Environments 

There  is  an  interesting  germ  of  an  important  idea  in  the  work  cited  above.  If  we  look  at 
th.  skeleton  strategies  of  Taylor  and  Lozano-Perez,  and  the  plan  checker  of  Brooks,  we 
see  an  underlying  design  philosophy.  Specifically,  strategies  and  environments  are  seen  as 
interwoven.  Given  a  robot  task  there  are  two  ways  to  proceed.  One  is  to  develop  a  strategy 
for  accomplishing  the  task  in  the  specified  environment.  It  is  in  this  direction  that  the  work 
on  uncertainty  has  evolved  over  the  last  few  years.  Another  direction  is  to  redesign  the 
environment  so  that  some  simple  off-the-shelf  strategy  is  guaranteed  to  succeed.  For  instance, 
one  might  develop  a  nominal  plan  for  accomplishing  the  task  under  the  assumption  of  no 
uncertainty.  In  order  to  ensure  the  plan’s  success  in  an  uncertain  world,  one  introduces  a 
new  sensor  not  originally  postulated  in  the  environment.  This  second  direction  for  dealing 
with  uncertainty  has  not  received  as  much  attention  in  recent  years.  An  important  research 
question  is  to  determine  the  conditions  under  which  redesigning  the  environment  is  a  possible 
solution.  Studying  the  information  requirements  of  manipulation  tasks  is  one  approach  to 
answering  this  question. 

7.10  A  Progression  of  Ideas 

This  proposal  builds  on  the  progression  of  research  discussed  above.  It  seeks  to  develop 
efficient  methods  for  planning  manipulation  strategies  in  the  presence  of  uncertainty.  We  see 
a  progression  from  symbolic  reasoning,  to  planning  in  the  presence  of  uncertainty,  to  designing 
sensors  and  environments.  At  the  heart  of  all  of  this  research  is  an  attempt  to  understand 
the  interaction  of  a  robot  with  its  environment  and  to  facilitate  the  automatic  programming 
of  robots. 
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8  Statement  of  Impact 

This  research  is  significant  on  three  levels: 

1.  First,  the  research  will  examine  directly  the  availability,  design,  and  use  of  existing 
simple  sensors  to  improve  the  reliability  and  effectiveness  of  assembly  operations. 

2.  Second,  the  research  is  aimed  at  breaking  the  uncertainty  barrier.  While  we  have  had 
formal  methods  for  dealing  with  uncertainty  in  robot  operations  for  about  a  decade,  the 
computational  complexity  of  these  methods  has  made  it  nearly  impossible  to  implement 
systems  for  dealing  with  uncertainty.  Fundamentally,  the  difficulty  lies  with  sensing 
uncertainty.  The  proposed  research  intends  to  circumvent  this  computational  complexity 
by  designing  sensors  that  provide  precisely  the  information  required  by  the  assembly 
system. 

3.  Third,  the  research  is  of  long-range  significance  in  understanding  the  information 
requirements  of  manipulation  systems.  Such  systems  include  factory  assembly  cells 
as  well  as  household  robots  or  humans.  Current  robots  are  extremely  limited  in 
functionality.  Additionally,  it  is  cumbersome  to  program  these  robots.  Much  trial  and 
error  testing  is  necessary.  We  simply  do  not  have  a  good  understanding  of  how  task 
strategies  can  fail  due  to  poor  modelling,  uncertain  control,  and  imperfect  sensing.  In 
order  to  automate  the  design  and  development  of  systems  with  broad  capabilities  it  is 
essential  to  understand  how  sensing  int^  ra,  „s  with  mechanics. 

We  envision  the  most  imir'  iiatc  impact  of  our  proposed  research  in  the  following  specific 
areas. 

•  Parts  Identification,  Sorting,  and  Orienting.  The  proposed  research  will  examine 
motion  and  sensing  strategies  for  orienting  small  parts.  A  typical  example  consists 
of  identifying  and  orienti’  *  parts  such  as  screws  and  bolts,  plastic  components,  and 
smau  subas  Mies.  These  parts  typically  arrive  semi-chaotically  at  a  robotic  workcell 
via  some  tra,  .port  system.  Existing  systems  for  orienting  such  parts  include  bowl 
feede.r,  specially  designed  nests  (such  as  Sony’s  APOS),  and  active  vision-manipulation 
strategies.  We  would  like  to  understand  the  design  of  easily  reconfigurable  special- 
purpose  parts  handlers  that  are  composed  of  simple  sensors  and  simple  mechanical 
g  ;s. 

•  Assembly.  Parts  identification  and  orienting  is  typically  a  prelude  to  an  assembly  task. 
The  proposed  research  should  be  of  interest  to  robot  programmers.  In  particular,  given 
an  assembly  task,  the  proposed  research  will  suggest  a  motion  strategy  along  with  a 
set  of  sensor  configurations  for  accomplishing  the  task  quickly  and  successfully.  Typical 
tasks  that  we  will  consider  are  insertions,  screwing  operations,  and  spring  loading. 

•  Task-Level  Planning.  Studying  the  information  requirements  of  robot  tasks  will  lead 
to  better  ways  for  programming  robots.  The  ultimate  goal  is  to  develop  task-level 
specifications  for  programming  robots.  To  date,  the  notion  of  a  task  is  not  even  well- 
defined,  given  the  problems  generated  by  uncertainty.  Exploring  the  information  flow 
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between  different  parts  of  a  task  is  crucial  to  developing  task-level  systems.  The  design 
of  action-based  special  purpose  sensors  is  particularly  important,  as  it  tells  us  what 
information  is  required  for  a  given  task.  Inverting  this  relationship  circumscribes  a  task, 
that  is,  it  provides  a  language  for  talking  about  tasks  in  high-level  terms.  For  instance, 
rather  than  talk  about  the  Screw  THE  Bolt  INTO  THE  HOLE  task,  one  might  refer 
to  the  task  by  its  sensing  and  motion  requirements:  “This  is  the  task  that  requires 
a  rotationally  varying  sensor,  moving  in  conjunction  with  a  rotational  motion  of  the 
arm.”  Both  descriptions  are  reasonably  high-level,  yet  the  first  conveys  no  information 
about  the  task.  The  proposed  research  will  help  in  developing  functional  task-level 
descriptions. 

•  Operation  in  Partially  Unknown  Environments.  The  proposed  research  examines 
the  design  and  construction  of  sensors  for  reducing  uncertainty.  Environments  consisting 
of  known  objects  in  unknown  locations  or  consisting  of  unknown  objects  of  fixed  type 
axe  conducive  to  this  approach.  For  instance,  we  imagine  designing  sensors  useful  for 
locating  and  sorting  recyclable  parts  such  as  those  found  in  consumer  trash  or  automobile 
recycling  plants.  Other  applications  include  sorting  and  locating  objects  in  hazardous 
waste  dumps. 
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